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Abstract 

As Institutional Research (IR) moves beyond its 
fiftieth anniversary, a new profession, called Institutional 
Effectiveness (IE), is emerging. In some respects, IR 
and IE are similar. IE, though, appears to be taking the 
leadership role. What are the structure, purpose, and 
responsibilities of IE offices? What are the implications 
for the IR field and its practitioners? How can Presidents 
and Provosts determine which of the two will best meet 
their institutions' needs for evidence-based decision- 
making, assessment, continuous improvement, and 
accountability? A decade ago, Peterson (1999) proposed 
that IR's future challenge would go beyond institutional 
improvement to helping facilitate its redesign and 
transformation. In the current climate, IR itself may be 
being redesigned and transformed. 



"IR is about reporting. We need more than that."This 
is what I was told by a self-study Chair at a university 
that had just completed its regional accreditation visit 
and was posting a position for a Director of Institutional 
Effectiveness. I had asked why they chose this route rather 
than expanding their institutional research (IR) office. 

I've heard this perception of IR expressed before, 
but until that exchange I had not connected it with 
the growing number of Institutional Effectiveness (IE) 
offices that are emerging at U.S. colleges and universities. 
Presumably, this growth was spurred by regional 
accreditors' emphasis on using evidence for continuous 
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improvement of student learning, or outcomes 
assessment, and increasing external demands for 
accountability such as, most recently, the Spellings 
Commission report. But why, I wondered, wouldn't 
IR be able to satisfy this need? Why would a new 
office and new profession be required? Is IR meeting 
this need at some institutions? Has it in the past? 
Where executives have decided to develop IE at 
their institution, has it replaced IR, added to IR, or 
has it subsumed IR? How are the IE department 
and its responsibilities organized in relation to 
IR? What are the implications for the institutional 
research profession and its practitioners? How 
can institutions best utilize existing — and new — 
resources to respond to the requirements of 
evidence-based decision-making, continuous 
improvement, outcomes assessment, and 
accountability? What structures and roles are best 
able to meet the institutional need? 

As a foundation for thinking about these 
questions, I reviewed websites and literature about 
the IR field. The web review included 30 Institutional 
Research and 30 Institutional Effectiveness websites, 
the AIR and international IR professional association 
websites, and regional accreditors' standards, 
statements, or use of the term institutional 
effectiveness. A Google search of IR and IE websites, 
using search terms institutional research and 
institutional effectiveness was the basis for locating 
the sites.^ The first 30 sites that met the following 
criteria were selected. First, the office or department 
was solely named IR or IE. Frequently, IR or IE is 
part of a composite office or unit title, such as 
"Institutional Research, Assessment, and Planning," 
or "Institutional Effectiveness and Analysis." By 
excluding these composite offices and focusing on 
singly designated IR and IE offices, the distinctions 
and commonalities between IR and IE can be more 
clearly seen. Second, only U.S. higher education 
institutions were included. Additionally, in the 
Google search for IR offices, large public universities 



primarily filled the first several pages. (Note: this 
did not occur when searching for IE offices.) To 
remedy what appeared to be a Google selection 
bias, other types of institutions shown in later pages 
were reviewed. This resulted in eight small (less 
than 10,000 headcount enrollment) and/or private 
colleges and universities being included in the IR 
web review. However, in 25 Google pages, only two 
community colleges appeared and both are system 
offices of very large institutions. They were included 
in the review. 

This paper is not an attempt at systematic 
research; it is intended to raise questions. For 
instance, although a search engine selection bias 
that excludes community colleges is feasible, it 
may be that community colleges are less likely 
than other institutional types to have offices 
that are solely named IR. Additionally, it may be 
that IE offices are more evenly distributed across 
institutional types, since large public universities 
did not predominate in the IE search results. I 
note these search results here because they raise 
the possibility that decisions about developing 
and utilizing IR and IE differ by institutional type. 
However, I have not attempted to investigate 
further, leaving the questions open for subsequent 
research. Instead of providing definitive answers, 
this paper is intended to promote reflection and 
discussion among IR and IE professionals and 
executives who must make the decisions about how 
to best fulfill these responsibilities at their campus. 

To lay the groundwork for discussion of the 
emergence of IE offices and the implications for 
IR, this paper first differentiates IR and IE roles and 
responsibilities. Subsequently, literature about the 
definition of IR and the state of the field at various 
points in time is utilized to put forth a macro-level 
view of the field's trajectory and suggest reasons 
why IR may not be meeting the needs addressed by 
IE offices. Utilizing results of the website review, the 
next section provides examples of IR and IE office 



' Data were collected from websites in December 2007 and January 2008. Additional details are available from the author. 
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missions, structures, and relationship to each other 
at multiple institutions. This examination is intended 
to reveal whether IR is being replaced by or 
complemented by IE. The discussion then moves to 
the question of whether IR, IE, or both are necessary 
to meet today's changing needs and the skills 
required to carry out IE responsibilities. In the last 
section, I consider implications for IR professionals 
and the IR field as IE offices gain ground. 

IR and IE: What's the Difference? 

While acknowledging the diversity of IR 
responsibilities at individual colleges and 
universities, the common core of IR is providing data 
and conducting research and analysis that supports 
the institution's enrollment goals, planning, 
assessment, program review, policy formation, 
and decision-making as well as accountability and 
external reporting (Howard, 2001 ; Knight, 2003; 
Saupe, 1990). External reporting and providing 
data to internal customers is a responsibility of all 
IR offices I reviewed. Following are two examples 
of common IR mission statements. Although many 
such mission statements are longer than these 
and delineate more specific types of studies, they 
encompass the same elements. 

University of /W/nnesoto; The Office 
of Institutional Research (OIR) at 
the University of Minnesota designs 
research studies and collects and 
analyzes data to provide information 
for institutional planning, policy 
formation, and decision-making. 

Among OIR's primary responsibilities 
is ensuring the integrity of the data it 
provides to University decision-makers, 
governmental agencies, and other 
internal and external constituencies. 

Missouri State University:TUe University 
has a 25-year heritage of institutional 
research activities and the Office 
of Institutional Research plays an 



influential role in many aspects of 
the University administration. Duties 
and responsibilities of the Office 
of Institutional Research include 
designing, analyzing, and presenting 
data and institutional research for the 
purposes of institutional planning, 
policy decisions, institutional 
marketing, and student and program 
assessment. The Office of Institutional 
Research is also responsible for 
reporting institutional data to various 
state and federal agencies. 

Most commonly, when IE mission statements 
differ from those of IR, it is by the role IE plays in 
planning, assessment, academic and administrative 
program review, and accreditation activities. IE 
personnel coordinate or develop these processes 
and/or monitor progress, assist faculty and staff in 
developing goals and objectives and evaluating 
progress, provide training, and document 
improvement. Mission statements for the Office 
of Institutional Effectiveness at Western Michigan 
University and University of Southern Mississippi 
are good examples of the responsibilities that most 
clearly distinguish IE from IR. 

Western Michigan University:To provide 
leadership to develop, integrate, and 
improve academic program planning, 
institutional and program assessments, 
and accreditation review programs at 
Western Michigan University. 

University of Southern Mississippi 
(USM):The Office of Institutional 
Effectiveness (IE) facilitates ongoing, 
integrated, institution-wide planning 
and evaluation processes. The staff 
of IE strives to be the University's 
source for assessment information, 
providing training and oversight 
of programs' assessment activities 
through annual assessment and 
periodic program reviews. IE is also 
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developing processes to analyze data 
with regards to the University's overall 
operations, providing information 
to lead departments across campus 
to make improvements. Institutional 
Effectiveness is dedicated to 
providing current, accurate, research- 
based guidance about assessment, 
program reviews, and institutional 
improvements. 

A separate IR office exists on both of these 
campuses. However, some traditional IR activity is 
evident in the USM IE mission (i.e., analyzing data 
for overall operations). This raises the question as 
to what characteristics distinguish the two offices 
at this campus. The following USM IR mission 
statement reinforces the scope of IR as stated in the 
previous IR mission statements. IR responsibilities 
stop at providing data and research. 

The Office of Institutional Research 
collects, archives, and maintains 
institutional data for the purpose of 
analyzing, distributing and presenting 
summary information. This information 
is used to support the decision-making 
process and the planning needs of 
all academic and administrative units 
within The University of Southern 
Mississippi. The Office of Institutional 
Research is responsible for reporting 
official data to the Board of Trustees 
for Institutions of Higher Learning 
for the State of Mississippi (IHL) and 
the Federal Government (Integrated 
Postsecondary Education Data System 
-IPEDS). 

With their often extensive knowledge 
of institutional operations and goals and 
methodological expertise, why wouldn't IR offices 
take on the coordinating, facilitating, training, 
process developing, and oversight roles that IE 
offices appear to be assuming? Why would a 
separate office, that sometimes overlaps IR in its 



analysis and data provision role (as illustrated by 
USM), be needed? Perhaps, it is the preference of IR 
professionals to limit their role. Or, perhaps the state 
of the IR field or IR's responsibilities or reputation on 
a campus leads executives to overlook or eliminate 
IR professionals from consideration when deciding 
how to meet this need. Based on the mission 
statements I reviewed and my knowledge of and 
experience in the IR field, I assume that, at present, 
IR is not widely taking on these responsibilities. The 
following section reviews this question historically 
and suggests possible reasons why. 

IR Yesterday and Today 

At its inception, IR appears to have had 
higher level responsibilities and to have reported 
higher in the organizational structure than it 
often does today. In some respects, those earlier 
responsibilities resemble today's IE. In fact, Sidney 
Suslow (1972) even used the term institutional 
effectiveness in his description of IR. 

According to Suslow's Declaration on 
Institutional Research (1972), "The institutional 
researcher's basic role encompasses the systematic 
appraisal of the higher education effort. The 
institutional researcher serves higher education 
and, in turn, his institution through critical 
appraisal and careful investigation of its processes 
and programs" (Section 1 , p. 2). He says its role is 
"evaluator of institutional programs" (Section 3, p. 

9). As such, IR identifies "programs that vary to an 
unacceptable degree from what was planned, spells 
out probable causes of the variance, and arrays 
alternative actions to improve implementation of 
the program or suggests changes in objectives" 
(Section 3, p. 9). IR identifies emerging needs not 
being met by existing programs and identifies 
programs that may be operating as planned but 
whose objectives no longer justify resources 
(Suslow, 1972). 

Somewhere between the mid-1 970s and 
1990, IR must have lost this breadth and level of 
responsibility, tending increasingly to focus more 
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narrowly on data provision and reporting functions. 
By 1 990, in his address at the AIR Annual Forum, 
Sherr called on IR to play a new role in helping 
colleges and universities improve the quality of 
their programs. In addition to "gathering, collecting, 
analyzing and disseminating information," IR 
practitioners should become "problem consultants" 
(p. 36), identifying problems and suggesting 
alternatives for improvement. IR "should and 
must play a leading role" in transforming 
higher education culture to one of continuous 
improvement (p. 44). Middaugh (1 990) too called for 
IR to go beyond counting and descriptions of inputs 
and "play a central supportive role" in institutions' 
introspective analysis of higher education processes 
(p. 1 90). Those that don't rise to this challenge, he 
predicted, will find themselves out of business. Yet, 
these calls to redefine IR identify as new activities 
that are similar to Suslow's 1 972 formulation. 
Assuming that IR did function as Suslow stated, why 
might the activities of IR offices have narrowed in 
the intervening years? 

Budget constraints of the early 1 980s resulted 
in downsizing and eliminating IR offices and 
decentralizing IR responsibilities (Middaugh, 

1 990). Simultaneously, the press to account for 
resources increased reporting requirements, and 
consequently may have directed the labor of a 
smaller staff to these tasks. In a 1996 survey of 
Presidents' perceptions of the effectiveness of 
institutional research (Harrington, Christie, & Chen, 
1996), respondents stated that underfunding and 
increased reporting requirements were barriers 
to effective IR. Nearly 20 years after Sherr and 
Middaugh's calls and more than a decade after the 
Presidents' acknowledgement of these barriers, the 
most common IR offices are one- or two-person 
shops (Volkwein, 2008), and reporting demands 
have not abated. 

Perhaps, at many institutions, IR has become 
overburdened with reporting and database 
management. A common lament among 
practitioners is that these requirements demand 



too much of their time. Suslow (1972) warned, "To 
conserve its energies, institutional research should 
not perform the functions of generator or maintainer 
of routine and extensive data files. . .The limited 
resources of any institutional research effort can be 
rapidly dissipated if that effort becomes burdened 
with repetitive, routine data collection and large 
systems maintenance. If these functions are called 
institutional research, they are incorrectly named" 
(Section 2, p. 7). Today, however, these functions, 
modified a bit by improvements in technology, seem 
to be accepted as part of IR. When combined with 
the extreme staffing limitations — under which so 
many IR departments labor — taking up the mantle of 
continuous improvement would be unlikely. 

IR's adaptation to the lack of staffing, 
reporting demands, and characteristics of some IR 
practitioners may have influenced this narrowing 
as well. For instance, downsizing and changes in 
responsibilities can require hiring staff with the skill 
sets needed to carry out modified tasks. Directors 
may hire for the ability to manage databases and 
accurately extract and manipulate data rather than 
statistical and evaluation expertise. Executives may 
hire Directors who can manage a technical office but 
are also expected to manipulate data themselves. In 
so doing, IR departments may have been reshaped 
to the extent that, when the desire or need for 
higher level analyses or recommendations arises, the 
department is incapable of responding. 

According to Presidents, IR is effective in 
multiple dimensions but is most effective at 
providing and reporting data. In Harrington, 

Christie, and Chen's survey (1996), Presidents rated 
their IR office most highly on accuracy. Ninety-six 
percent agreed that their IR department provides 
accurate data and information. A little more than 
90% agreed IR is effective at reporting. However, 
Presidents want more than this. Ninety percent 
expected analysis and interpretation to be part of 
IR reports. Twenty-nine percent urgently wanted 
recommendations with the report. Only one-third 
of Presidents did not expect recommendations from 
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IR.This study does not report how often Presidents 
get such information from IR. Provosts too expect 
IR to go beyond data collection and provide this 
type of high-level assistance (Okigbo, 2008). That IR 
provides data, rather than information, is a common 
complaint I hear about IR from users of its services. 
IR customers often do not know what the data 
mean or how to use them. They need and want data 
converted to information for their decision-making. 

To the extent that interpretation is not provided, 
it could be the result of reporting demands and 
time constraints, lack of access and expertise of 
IR staff at some institutions, or adherence to an IR 
professional value. Interpreting data and making 
recommendations is more time-consuming and 
requires greater knowledge of the institution 
and the issue at hand than does producing and 
disseminating data tables. Consequently, both 
the time and expertise necessary to perform this 
function may be increasingly limited by the growth 
in reporting demands. Additionally, many IR 
professionals express concern that they are not part 
of the ongoing information flow. To the degree that 
this is the case, they cannot develop the contextual 
knowledge needed to make interpretations and 
recommendations (Chambers & Gerek, 2007) . 

Some IR practitioners resist interpreting data 
on the grounds that it compromises objectivity. 

The boundaries of the IR practitioner's role have 
been debated since the profession's inception. 

In his presidential address. Lasher (1983) noted 
IR professionals' desire to remain apolitical 
as a Justification for providing data without 
interpretation. He says data become political 
when they are selected, organized, analyzed, and 
interpreted. Without these processes, however, 
data are practically useless. Lasher considers 
IR professionals to be in a unique position to 
influence an institution's direction by virtue of 
their ability to provide timely, useful management 
information. He says they must do so for IR to be 
vital. Suslow (1 972) goes even further by saying that 
practitioners who have conducted objective study 



and research should feel free to state preferences 
for alternatives or to make recommendations based 
on that research. As noted. Presidents, too, say 
they want interpretation and recommendations 
from IR. Harrington, Christie, and Chen's survey 
(1996) reveals a related characteristic. Fifty percent 
of Presidents said they perceive IR as reactive. 

Nearly 90% said they prefer IR to be proactive. 
Interpreting data and making recommendations is a 
proactive activity and requires initiative. Executives 
perceive one of IR's barriers to effectiveness to be 
IR professionals' lack of initiative. As Lasher (1 983) 
cautioned, remaining on the sidelines is the way to 
make IR expendable. 

Changes in responsibilities and influence 
may have been affected by IR's location in the 
organization as well. As previously mentioned, 
a common complaint among IR Directors is that 
they are not privy to information they need to be 
effective. IR started by reporting to the President. 
According to Taylor (1 990), over time, it moved 
down in the organization as institutions grew in size 
and complexity and the President's role became 
external rather than one of internal management. 

To remain focused on the internal workings of the 
institution, IR began to report elsewhere. Most 
commonly, that shift in reporting moved to the 
Provost. However, any further down in the reporting 
structure, Taylor says, moves IR too far out of 
the routine information flow to gain the needed 
contextual information to perform effectively at 
a high level and makes it difficult for IR staff to 
influence decision-making. 

Location in the organization may not be the 
only reason IR managers feel they lack needed 
information. The culture of the institution and 
personal style of the executives can impact this too. 
But location can subject IR to the old adage, "out of 
sight, out of mind." 

The possibility that IR is, in many cases, isolated 
and removed from the heart of the institution 
is suggested by my website review. Most solely 
named IR offices appear to be freestanding, rather 
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than part of an umbrella unit that encompasses 
other "effectiveness-type" departments such as 
assessment or strategic planning. Noticeably, a 
little more than two-thirds of these websites did 
not link to their institution's strategic plan or an 
office that does strategic planning, or to assessment 
plans or an office that does assessment or any 
other related functions or responsibilities. To the 
extent that these offices and documents exist on 
those campuses, this absence of linking suggests to 
me that these IR offices are operating in isolation. 
Whether what actually happens in practice and 
whether the website adequately reflects that 
practice cannot be determined by such an external 
review. However, given the perennial complaint, 
within our profession and among accreditors, that 
people do not use data, make meaning from them, 
and then make change based on those data — then 
to the degree that this isolation exists, IR's lack of 
influence should not be a surprise. These IR websites 
do not show that IR is central to the institution, or 
even engaged with it. As stated above, it has been 
argued that such distancing is important to assure 
objectivity. The danger, however, is irrelevance to 
the core mission and vulnerability to downsizing. 
Standing apart does not put one in the path that 
leads to new or greater responsibility. 

If IR today does not widely perform the role 
Suslow, Sherr, and Middaugh expound, it may be 
due to increased requirements for data reporting 
and management, inadequate staffing, budget 
cuts, organizational alignments that make the 
role unfeasible, or some of its own professional 
practices. But whatever the reasons, in the context 
of accreditors' increased push for continuous 
improvement and evidence-based decision-making, 
IE offices are developing, and a new profession is 
emerging to take on some of this responsibility. 

IE Office Mission, Structure, 
and Relationship to IR 

Based on the descriptions shown in mission 
statements, both IR and IE potentially work across 



all divisions of the institution and in numerous 
areas. Both have an analysis role. At some 
institutions where both IR and IE offices exist, 
their missions overlap — usually in the realm of 
data provision. For both, the boundary appears 
to be where institutional decision-making and 
implementation takes hold. At that point, both IR 
and IE practitioners' roles end. 

Within these commonalities, the breadth of 
IR and IE may be perceived differently. IR, as the 
primary institutional data provider, conducts 
research and data analyses for multiple purposes, 
not Just improvement. IR work can include 
projections, budget analyses, workload studies, 
and other research that helps facilitate operations, 
develop policy, or improve programs. In this 
respect, IR appears broader than IE. On the other 
hand, IE appears broader than IR in that its role may 
involve assisting in establishing goals, advising 
on implementation of improvement initiatives, 
planning, facilitating, coordinating, oversight, and 
training in addition to data analysis. 

IR and IE relationships. How these 
responsibilities are viewed vis-a-vis each other 
may"determine"the IR and IE office location and 
relative influence in the organization, or that 
configuration may "reflect" their relationship 
to each other. For instance, if IE is perceived as 
broader than IR, IR may be a support service 
to IE. If IR is perceived as broader, IE may be a 
department within or activity of IR. Or, they could 
be viewed as equivalent, horizontal processes 
but separate departments. Alternatively, an 
institution could choose to have only an IR office 
or only an IE office that encompasses both sets 
of responsibilities. In my website review, I found 
all but one of these structures and relationships: 

IR that encompasses most of the activities of IE. 
Although such an arrangement may exist, in no 
case did I find instances where IR appeared to be 
conducting training, facilitating, or coordinating 
IE processes. In cases where the term coordinating 
was used in IR mission statements, it tended to be 
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associated with external reporting or extracting, 
defining, or utilizing data and research between 
internal departments. Additionally, I found only 
one instance in which IE was a department 
within a unit called IR. In that case, nothing in the 
office description suggested that either IR or IE 
is developing processes, facilitating, or training. 
Both appear to be providing data and research to 
support IE-type functions. 

At some institutions, an IE office exists instead 
of an IR office and appears to carry out only IR-type 
activities (e.g., at the University of West Alabama 
and Galveston College, respectively). 

University of West Alabama: The 
Office of Institutional Effectiveness 
is responsible for a comprehensive 
program of data collection, analysis, and 
reporting for the University community. 
The office also prepares both routine 
and special research reports for the 
administration, faculty and staff, and 
provides assistance to faculty and staff 
in their research efforts." 

Galveston College: The mission of the 
Institutional Effectiveness office is to 
support planning, decision-making, 
and assessment at Galveston College 
by providing information that is timely 
and of the highest quality. The Office 
of Institutional Effectiveness (IE) is a 
service oriented unit that provides 
information and proposes alternative 
solutions to support the decision- 
making process. The scope of the 
unit's responsibility is college-wide. 

The unit provides official reporting to 
external agencies, the Texas Higher 
Education Coordinating Board, and the 
entire College community including 
the Board of Trustees, and other 
academic and administrative units. 

The Institutional Effectiveness office 
also assumes the responsibility of 



either generating or supervising the 
development of all official University 
data reports. 

These IE offices appear to be IR by a different name. 
Whether an IR office previously existed but was 
renamed or whether an office responsible for IR- 
type activities began at these campuses under the 
name IE cannot be determined from the websites. 
Regardless, this type of arrangement can be viewed 
as IE replacing IR. 

Some IE offices include IR activities but also 
some degree of assessment, accreditation, or 
planning oversight and coordination activity. The 
extent of that activity is difficult to determine 
from websites but examples include Coahoma 
Community College and the University of 
Richmond. At these campuses, no office specifically 
named IR exists. Instead, it is called IE. However, 
it appears that, at some point, the University of 
Richmond must have had an office called IR because 
the IE office URL is http://oir.richmond.edu. Again, in 
cases such as these, IE has replaced IR. 

IE at Eastern Kentucky University (EKU) is 
an example of IR being subsumed. Until about 
2002 or 2003, supporting outcomes assessment 
was part of IR's responsibilities at EKU. An IR staff 
member served on the University Assessment 
Committee and provided training to the faculty on 
developing learning objectives, selecting measures, 
and utilizing results; however, this arrangement 
did not last. While gearing up for their SACS 
reaccreditation, coordinating strategic planning 
and managing the accreditation process fell to a 
faculty member. Eventually, it was determined that 
a full-time administrative position was needed to 
handle the ongoing responsibilities rather than 
adding on to a faculty load. An Executive Director 
of Institutional Effectiveness and Research was 
hired to fill this need. An IE office was formed, and 
the IR staff member who had been supporting 
assessment became the Assistant Director of IE. The 
IR Director then began reporting to the Executive 
Director. Previously, the IR Director had reported to 
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the Provost (S. Street, Assistant Director, personal 
communication, August 27, 2008). 

At institutions where separate IR and IE offices 
exist, they may be competitors, collaborators, or 
horizontally located independent departments. For 
example, at Western Michigan University there is an 
Institutional Effectiveness (IE) office and an office 
called Student Academic and Institutional Research 
(SAIR). Both are linked from the Provost's website and 
both are headed by Vice Provosts. However, neither 
the IE site nor the SAIR site links to the other — 
suggesting either independence, competition, or 
both. The IE Mission statement reads: 

To provide leadership to develop, 
integrate, and improve academic 
program planning, institutional 
and program assessments, and 
accreditation review programs at 
Western Michigan University. 

The SAIR mission statement, paraphrased here due 
to its length, states that it guides and coordinates 
university strategic planning and provides data and 
information for administrative decision-making 
and conducts external reporting. The missions 
seem to clearly distinguish the two, suggesting 
independence. However, one of the SAIR staff is an 
Assessment Programs Specialist, and SAIR's mission 
statement ends with "The Office relies on and works 
closely with other parts of the University that collect 
data used as the basis for Student Academic and 
Institutional Research's reports."This statement and 
the staff title may indicate collaboration through 
intentional overlap, but given that no specific 
department with which SAIR works closely is named 
or linked, these artifacts could instead reflect 
competition or duplication of effort. The degree 
to which IR and IE departments with this type of 
organizational alignment collaborate, compete, 
ignore, or serve one another is likely dependent on 
internal dynamics and individual relationships. 

In some cases, both IR and IE are departments 
within a larger unit. This is the structure at Indiana 
University Purdue University Indianapolis (lUPUl) 



where the umbrella organization is called Planning 
and Institutional Improvement (Pll). IR and IE are 
clearly defined departments within Pll, which also 
includes three other departments. The departments 
appear to be horizontally located within the unit. 

In configurations such as this one, the IR and IE 
relationship may be considered collaborative or 
customer/supplier, again depending on internal 
dynamics. 

Yet another method of organizing is 
represented by Tidewater Community College. 

There, Institutional Effectiveness is the name of a 
unit that includes two departments. One of those 
departments is Institutional Research, and the 
other is Student Outcomes Assessment. In this 
arrangement, IE is the overarching title. The unit 
manager is the Director of Institutional Effectiveness, 
who has an Assistant Director of IE. Other positions 
in the office are designated with a title followed 
by the qualifiers Student Outcomes Assessment or 
Institutional Research. This arrangement clearly 
elevates IE above IR. Like Outcomes Assessment, 

IR appears to be a tool in service of institutional 
effectiveness in its broader, process sense, rather 
than in the sense of IE as an office. I would expect 
this arrangement to facilitate collaboration. 

In my web review, I rarely saw the term 
institutional effectiveness mentioned in IR mission 
statements although activities that might be 
considered components of IE were frequently 
named (e.g., strategic planning, outcomes 
assessment, and program review). Where IR mission 
or responsibilities statements did specifically 
include institutional effectiveness or continuous 
improvement, the IR role is supporting IE by 
providing data, information, or methodological 
consultations to individuals engaged in effectiveness 
activities or the staff of an office called IE. 

Whether IE will replace, subsume, or work 
hand-in-hand with IR is too early to tell. As shown, 
examples of each of these scenarios exist. In 
addition, while IE offices are emerging, IR staffing 
has grown as well (Volkwein, 201 1 ). Given this 
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finding, perhaps IR expansion, rather than IE 
formation, is the path some institutions are taking 
to meet continuous improvement needs. Or, 
possibly this IR growth is responding to other needs 
such as reporting, accreditation, or IE office support. 

IR, IE, or Both — What Best Meets 
Changing Needs? 

With the increased calls for accountability, 
continuous improvement, assessment, and 
evidence-based decision-making, often in the 
context of shrinking budgets, colleges and 
universities need to utilize the resources at their 
disposal in the most effective way. As such, 
they are grappling with how best to respond to 
these external pressures and requirements and 
considering what structures and roles are most 
likely to meet the need. 

Develop a new office or expand an existing 
one? How would one decide what option would 
best utilize resources while most effectively meeting 
institutional need? According to Nichols and 
Wolff (1 990), developing a new office with a new 
name keeps perceptions of an existing office from 
affecting the new activities and avoids assigning 
responsibilities to an existing staff that likely does 
not possess the needed skills. With a new name, 
a clear focus for the office can be established, 
and new expectations can be clearly defined and 
communicated. The office can be strategically 
positioned where it would be most influential in 
carrying out its activities, if the current office is not. 

Expanding an existing office takes advantage 
of an infrastructure that is already in place (Nichols 
& Wolff, 1990). Consequently, start-up time may 
be quicker and initial cost lower than developing 
a new office. Additionally, it would build on the 
institution's existing knowledge-base. If the 
existing office reports to Academic Affairs and 
works effectively with academic units, and the new 
responsibilities are to include outcomes assessment 
and program review, change may be more smoothly 
and easily incorporated. 



Modifying the title of an existing office to reflect 
its expansion or new direction is an option, as is 
joining a new and existing office, or multiple existing 
offices, into a collaborative unit. Hybrid offices, 
or units, with titles such as Institutional Research, 
Assessment, and Planning; Institutional Effectiveness 
and Analysis; or Institutional Effectiveness, Planning, 
and Research combine these functions. In fact, such 
combinations are common (Saupe, 1990). In some 
cases, these configurations seem only to reflect 
that data and information are provided for these 
purposes. In others, it is an attempt to bring together 
complementary functions and develop an integrated 
approach to planning, continuous improvement, and 
accountability. However, this option is not a solution 
that can avoid additional resource allocation. Such 
a merging may eliminate unnecessary duplication 
of staff, tasks, or physical space, if such exists, and 
may promote efficiencies and allow allocation shifts, 
but almost certainly there will be gaps that must 
be recognized and addressed to assure the new 
structure serves its purpose effectively. 

Either choice involves infrastructure, integration, 
and staffing issues that must be explicitly anticipated 
and addressed. For instance, one quandary is how 
to integrate responsibilities that require different 
skill sets, tasks, and perspectives. Offices may 
work at higher or lower levels of the organization 
with different clientele. IR responsibilities are 
typically institutional-level while student outcomes 
assessment may be focused on departments. 
Consequently, IR may work with fewer and higher 
level administrators and assessment with a larger 
number of mid-level managers, faculty, and line 
staff. IR work is predominately with computers and 
data while assessment and planning work includes 
a considerable amount of discussion. One group 
requires more solitude and concentration while 
the other needs considerable phone and in-person 
interaction that can be disruptive if housed in the 
same space. Consideration must be given to whether 
these functions should be separate physically or in 
the reporting line or how they could be successfully 
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joined in a single unit. Infrastructure and integration 
issues are numerous and more complicated with 
mergers than when developing or transitioning a 
single department. 

Understanding the skills needed to perform 
new responsibilities effectively is critical because 
personnel who possess the requisite skill sets 
must be hired or existing staff must be trained. 

If choosing the latter, existing responsibilities 
must be transferred to allow time for training and 
taking on new tasks. Additionally, it is possible that 
existing staff possess the needed skills but have 
been underutilized in this respect. This situation too 
would require transferring existing responsibilities 
in addition to accommodating role shifts. Adding 
such responsibilities to an existing office without 
increasing its capacity or recognizing and modifying 
structural issues is untenable. It can overstretch staff, 
leave responsibilities unfilled, and create stress and 
morale problems that lead to employee turnover. 

To assure that the chosen configuration has 
adequate authority and access to information to 
be effective, the required level of management 
should be carefully considered in light of the 
institution's culture. Based on my web review, 
there are more management positions that are 
higher than Director-level in IE than in IR. IR 
was most commonly led by a Director. IE too is 
frequently led by a Director, but less often than IR. 
Associate Provosts or Associate Vice Presidents are 
more commonly found in IE than IR. In the web 
sample, only 23% of IR management positions 
were Assistant Vice President or higher compared 
to 43% of IE management positions. For some IR 
professionals with the requisite skills, IE may be a 
remedy to the often-heard complaint about IR's 
short career ladder (Johnson, 1 982; Knight & Leimer, 
201 0). Advancement could mean moving out of IR 
and into IE. Such a move could offer new challenges 
and breadth of tasks and could broaden one's skills. 

Skills needed for IE. What skills are needed to 
work effectively in IE? That may depend on the level 
of employment and the specific tasks assigned to IE 



at a particular institution. Assuming a definition of 
IE that encompasses student outcomes assessment, 
program review, and accreditation, the following 
are necessary to varying degrees: research methods 
and statistical skills; knowledge of accreditation 
requirements and assessment methods and 
processes; strong leadership skills; the ability to 
facilitate groups, build consensus among multiple 
constituencies, negotiate, communicate in non- 
technical language, coordinate people and projects; 
and, at the management level, management 
skills that go beyond directing a technical staff. 

A recent collective contribution (on the ASSESS 
listserv) to the question about skills needed to 
work in assessment compiled a list barely short of 
walking on water: sensitivity, open-mindedness, 
flexibility, listening, enthusiasm, commitment to 
learning, sense of humor, ability to see another's 
perspective, ability to build others' self-confidence, 
creativity, team-building, problem-solving, a thick 
skin, tolerance for ambiguity, patience, ability to 
motivate others, ability to teach, ability to build 
trust, and the ability to use data to tell a compelling 
story. Technical skills, though assumed by some of 
the contributors, did not even make the list until 
the oversight was pointed out. Once it was, some 
participants still downplayed this skill set in favor of 
interpersonal abilities. 

A brief review of job postings for institutional 
effectiveness management positions included 
institutional research and outcomes assessment 
experience, knowledge of and/or experience with 
accreditation processes, knowledge of research 
and statistical methods, experience with statistical 
software and database management, analytical 
skills, organizational, project management 
and facilitation skills, leadership skills, strong 
interpersonal and written and oral communication 
skills, and demonstrated ability to work effectively 
with faculty and administrators. From personal 
experience managing a blended, or hybrid, office 
that includes some of the elements of IE offices, I 
add an additional requirement also pointed out by 
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Nichols (1 990): high tolerance for the visibility that 
comes with a high-profile position on campus. For 
institutional researchers used to laboring in relative 
obscurity, this can be a big adjustment. 

IR practitioners possess this knowledge and 
many of these skills, especially the technical and 
analytical skills. The interpersonal, management, 
and leadership skills are not as common in IR Job 
postings, yet demonstrating these abilities may 
help advance one's career and be a valuable tool in 
elevating IR from a technical to a leadership level in 
the organization. But how can one gain these skills? 

Currently, experienced Assessment Coordinators 
or Directors are few and colleges and universities 
are often best served by growing their own. The 
field may still be new enough to learn assessment 
on the job, thereby broadening one's skill set. 
Assessment conferences and workshops are 
numerous throughout the year and all across the 
U.S. AIR began an Assessment Institute in 2008, and 
the AIR Forum includes tracks on assessment and 
academic programs. Webinars too are a way to build 
one's assessment knowledge. As for leadership skills. 
Deans, Provosts, Vice Presidents, and Presidents 
benefit from training and mentoring programs such 
as those offered through the American Council for 
Education (ACE) and Harvard University, but I know 
of none that are specifically targeted to staff or 
managers of IR or IE. Some schools of business offer 
manager and executive workshops or short-term 
classes for a general audience that may be useful. 
For women in higher education administration. 
Higher Education Resource Services (HERS) may be 
an option. 

I know of no conferences on institutional 
effectiveness or professional organizations 
dedicated to IE. I can find venues for IE components: 
outcomes assessment organizations and 
conferences, planning through the Society for 
College and University Planning (SCUP) — though 
its focus has been primarily physical facilities 
rather than strategic or academic planning — 
and accreditation conferences, workshops, and 



networking via the accreditation associations. 

AIR Forum tracks cover a wide range of higher 
education issues, including assessment and 
planning. Another track mentions organizational 
management and change, topics that may be 
helpful to executives deciding to institute IR, IE, 
or some combination as well as those charged 
with the implementation. But, the Forum happens 
once per year, sessions are short and, on some 
topics, sparse, and quality varies. In my web 
review, I noticed that the European AIR has a 
different mission than all other IR professional 
associations. It is an association for professionals in 
higher education management and for academics 
researching management issues. Maybe we can 
learn something from the European AIR that could 
be applied to our changing circumstances and 
needs. Such knowledge may be helpful to those 
IR managers who have been given responsibility 
for assessment, planning, or accreditation and are 
wondering how to integrate and manage it all. 

Implications for IR Practitioners 
and the Profession 

IR professionals who are concerned about 
higher education, the IR profession, and our own 
career advancement and opportunities, should 
pay attention to the emergence of IE. For some 
executives deciding how to meet continuous 
improvement and accountability needs of 
colleges and universities, IR is not the preferred 
choice. Evidence exists that some institutions are 
supplanting IR by giving the title IE to traditional IR 
activities, or subordinating IR to a new IE office. 

Some might consider a departmental title 
change to be cosmetic, and it may well be at some 
institutions. In those cases, perhaps the change 
is inconsequential. It may be only a sign of the 
times (i.e., accreditation's emphasis on institutional 
effectiveness gets reflected in a name). However, 
what if there is more to the name change than that? 
Perhaps it reflects new institutional priorities or the 
need for a fresh approach or application of different 
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skills than those traditionally associated with IR. 
Maybe it is an attempt to disassociate the activities 
from the perception of IR as reporting, as noted in 
the example of the self-study Chair at the beginning 
of this paper. How is the IR field perceived outside of 
the field? Not so much in terms of how effectively it 
carries out its responsibilities, but what its roles and 
responsibilities are. 

The general perception of a field can help or 
hinder anyone in that field regardless of his or her 
individual circumstances. Our salaries, the length 
of the career ladder, and access to challenging 
opportunities are determined at least in part by 
perceptions of characteristics of practitioners in a 
field and the field's perceived value. Consequently, 
the subordination of IRto IE may be of greater 
concern than the renaming of IR to IE. In some 
cases, what are coming to be defined as IE 
responsibilities have considerable overlap with 
IR responsibilities, yet the leadership or executive 
functions appear to be assigned to IE offices. When 
IR and IE offices exist at the same institution, both 
are often located within Academic Affairs. Given 
this location, and the fact that an IR office existed 
before an IE office, why would IR not be given the 
leadership responsibilities rather than developing 
another office or profession to assume them? 
Without these responsibilities, IR risks becoming 
increasingly defined by its reporting and data 
support functions, especially given increasing 
reporting requirements such as those of the Higher 
Education Opportunity Act. 

As IR moves further away from executive 
decision makers by the insertion of a superordinate 
departmental structure, the possibility that IR can 
be the evaluator of institutions, as Suslow described, 
is diminished or extinguished. To perform this role, 
contextual knowledge that can only be gained 
through being part of the high-level information flow 
is essential. If that access is given to a new profession 
or office, IR's influence wanes. Ultimately, that can 
affect its perceived value to the organization. 



Given these potential consequences, it is 
important to know how the decision to start IE 
offices is being made. Are any perceptions or 
experiences of IR contributing to these decisions? 
Are decisions being made in a way that IR can 
influence them? Perhaps by making targeted 
changes in hiring, training, or allocating time or 
resources differently? Without knowledge of the 
decision-making process, we cannot anticipate 
IR's trajectory — whether IR is a profession headed 
toward obsolescence, one that will need to compete 
with a similar office in times of budget cuts, or 
whether it is on sure footing even if perceived as 
less central to decision-making. 

Ways to meet these challenges should be 
discussed within the IR field. More research into the 
IE office trend is needed, whether AIR sponsored or 
taken up by individuals on their own. AIR advocacy 
to higher education executives may be beneficial 
toward securing adequate IR funding, institutional 
placement, and dismantling of other institutional 
barriers that would facilitate IR's role as evaluator of 
institutional processes and programs. Professional 
development for IR practitioners, especially 
managers and analysts who desire to advance in 
the field, should move beyond technical training 
to leadership skills that can help them proactively 
plan and negotiate for a strong, healthy IR at every 
institution where IR exists. 

Organizing for continuous improvement, 
assessment, accreditation, and accountability can 
be costly. Executives are grappling with how best 
to meet the need. A decade ago, Peterson (1 999) 
said IR's upcoming challenge may surpass helping 
institutions improve. Instead, IR will help facilitate 
"institutional redesign and transformation" (p. 

84). In the current climate, IR itself may be part 
of this redesign and transformation. We can rise 
to Suslow's declaration, even move beyond it to 
Peterson's transformative stance. But doing so 
requires leadership, from each of us and from our 
professional association. But after all, leadership is 
what Provosts and Presidents are asking of us. 
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